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We study the effect of interactions on the properties of a model 2D topological Kondo insulator 
phase. Loosely motivated by recent proposals where graphene is hybridized with impurity bands 
from heavy adatoms with partially filled d-shells, we introduce a model Hamiltonian which we believe 
captures the essential physics of the different competing phases. We show that there are generically 
three possible phases with different combinations of Kondo screening and magnetic order. Perhaps 
the most dramatic example of many-body physics in symmetry protected topological phases is the 
existence of the exotic edge states. We demonstrate that our mean field model contains a region 
with a time-reversal invariant bulk phase but where TR symmetry is spontaneously broken at the 
edge. Such a phase would not be possible in a non-interacting model. We also comment on the 
stability of this phase beyond mean field theory. 


I. INTRODUCTION 

The discovery of topological insulators in “inverted” 
semiconductors with strong spin orbit coupling led to an 
explosion of interest in topology in condensed matter sys¬ 
tems. The combination of topology and strong correla¬ 
tions of electrons is a subject of major current activity, 
and systems in which both are at play suspected to host a 
variety of unique phenomena.® “Heavy fermion” and re¬ 
lated materials with heavy lanthanide elements are nat¬ 
ural places to seek such phenomena, as the electrons on 
these atoms experience strong spin-orbit coupling (SOC) 
- which is a driving force for non-trivial topology in many 
systems - and are strongly correlated. Theory recently 
suggested a concrete role for topology in these systems 
in the form of Topological Kondo Insulators (TKIspl2l. 
A TKI is a Kondo lattice system involving rare earth 
ions which at low energies hybridize with lighter conduc¬ 
tion electrons forming a topological insulator. SmBg is 
strongly suspected to be such a TKl®2J. 

While SmB 6 and indeed any Kondo lattice system 
is indisputably a strongly correlated electronic system, 
the low energy description of a TKI in terms of effec¬ 
tive bands seems indistinguishable from that of an un¬ 
correlated topological insulator. This is disappointing 
in view of the hope for new phenomena in correlated 
strong SOC systems. The aim of this article is to con¬ 
sider the possibility of competing states, and seek out 
new effects arising from electronic correlations in a sys¬ 
tem which may host a TKI. We do this in the context 
of a model motivated by the recent proposal of realiz¬ 
ing a TKI in graphene doped with heavy adatoms. In 
this model, described in detail in Sec. we obtain a 
zero temperature phase diagram which embeds the topo¬ 
logical insulator physics into the classic Doniach phase 
diagranPfor Kondo lattice systems, with its magnetically 
ordered and Kondo screened phases. The magnetic or¬ 
dering quantum phase transition is discussed in this con¬ 
text. We also consider the characteristic behavior of the 
boundary of the TKI, and show that this model is prone 


to surface magnetic ordering , even within the TKI state. 
Such spontaneous and intrinsic time-reversal breaking at 
the surface of a TKI could be the desired hallmark of 
correlations in the TKI state. 


A. The Model 

We study a tight-binding model of graphene, with a 
localized d-orbital electron site at the center of each face 
on the honeycomb lattice. Such a model was studied ex¬ 
tensively in [shu], via a combination of first-principle 
calculations on a tight binding model and density func¬ 
tional calculations. There it was shown that the strong 
onsite spin-orbit term for the localized d-electrons, when 
hybridized with the conduction electrons, c, conspire to 
create a band insulator with a nontrivial topology. This 
topological phase on graphene is reminiscent of the orig¬ 
inal proposal of topo logical order in graphene by Kane 
and Mele in 200tP®-^l Our goal is to study explicitly the 
effect of interactions on such a model. 

In [S], the DFT calculations show that the most im¬ 
portant angular momentum states of the d electron are 
those with L z quantum number m = ±1. These arise 
from the d xz and d yz adatom states, and so we restrict 
our model to states with these angular momenta. We in¬ 
clude a spin-orbit coupling term for the d-electrons, but 
not for the conduction electrons where the small mag¬ 
netic moments are expected to lead to a negligibly small 
amount of spin-orbit coupling. The chemical potential of 
both the c and d electrons is set so that there are two 
c and two d electrons per unit cell. This is a necessary 
condition if the hybridization is to lead to a band insu¬ 
lator. 

The complication comes when we include an interac¬ 
tion term between the local d moments and the nearby 
conduction electrons. In order to write down this interac¬ 
tion, need to know the spin state formed by the compos¬ 
ite two-body state sitting at each d-site. We assume that 
within the m = ±1 subspace, a Hund’s rule type coupling 
makes it energetically unfavorable for both d-electrons to 
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have the same angular momentum quantum number. We 
therefore ignore the possibility of the d-electrons forming 
spin-2 states, |+ t> + -l) and |— ti — I), when writing the 
form of the interaction. The remaining states are 

Stot = 0 : |+tn — 4-) — — T) 

&ot = l : |+t,-t> 

1+ ti — 4-) + 1+ 4 -j — t) 

1+1)— I)- 


Within this subspace the Kondo interaction is a spin-spin 
interaction which occurs between the S t ot = 1 states, 
while the S.O. coupling term favors the |+ l, — f) state, 
which is a linear combination of the singlet and triplet 
S z = 0 states. 

We define the operator Ci p a , which is a linear combi¬ 
nation of Ci c , operators which carry angular momentum 

p = ±1. 


C R , 


P,(T 


_L V'' p -*(ir/3)p(j-l) 

V6 2 - R+e 

j=i 


E ( V p {k)c A ,k,a + V* p (k)c B ,k,c t) 

k 


( 1 ) 


where A and B denote the two sublattices of the hon¬ 
eycomb lattice, and are the nearest-neighbor lattice 
vectors connecting the c and d sites. 

We can now construct a spin-1 operator for both the c 
and d electrons from two spin-A operators as follows, 


El dln^&apdm'P 

771=±1 

(2) 

y 1 ^'m,a <J apC m ,p, 

(3) 


m=± 1 


where a is the usual spin-1/2 Pauli vector. The Kondo 
interaction is then an antiferromagnetic spin-spin inter¬ 
action between the spin-1 particle on the d-sites and the 
effective spin-1 formed by a linear combination of the con¬ 
duction electrons near this d-site. The 4 fermion Kondo 
interaction is therefore 

Hk = J E Si ,i ■ si ,j. (4) 

i 


We also include an RKKY spin-spin interaction in our 
model. This is an interaction term between the d-electron 
states on different sites which is mediated through con¬ 
duction electrons. Performing second order perturbation 
theory in the Kondo interaction, the d-spin on one site 
interacts antiferromagetically with the conduction elec¬ 
trons near that site which in turn act antiferromageti¬ 
cally with a neighboring d-site. We assume this creates 
an effective ferromagnetic interaction between the two d- 
sites and parameterize this term with the variable J m . 
Although in principle the RKKY interaction is included 
in Eq. Q, it is difficult to derive both the Kondo and 


RKKY effects togetheJ 14 * 15 ! Therefore, in our model we 
include as a separate parameter the RKKY interaction 

Hrkky = ■ Sij. (5) 

(*i> 

Therefore, the simplest interacting model that we be¬ 
lieve captures the essential many-body physics of the TKI 
system is 

H = 'y ] £kc\ck + A y ' (d\ pa .di p(T — 2) + \x y ] ( c jg c i^ ~ 2) 

fc,cr i,p,i7 i,t7 

+A^(4 a (T «/3 d + / 5 ~ d-a^apd-p) 

H-J E $l,i ’ ^1,2 + Jrn E^M 'Si, ( 6 ) 

* (»i> 

The goal of this paper is to examine at the mean field 
level the effects of the two interaction terms. 


II. MFT AND THE BULK PHASE DIAGRAM 
A. Outline 

In this section we will describe the general phases 
which exist in our model. This allows us to embed the 
TKI phase within the standard phase diagram for the 
Kondo lattice^. We show that there is in general a first 
order phase transition from a phase with no Kondo order 
(where the d-spins order magnetically) to a TKI phase 
at some nonzero value of the Kondo exchange J. Fur¬ 
thermore, this phase can be destroyed by RKKY type 
spin-spin interactions which are present in the fully in¬ 
teracting theory and so we include as a separate term in 
our mean-field Hamiltonian. If the system orders mag¬ 
netically, time-reversal symmetry is broken and any dis¬ 
tinction between the topological phase and a trivial in¬ 
sulator loses meaning. A similar MF phase diagram was 
calculated in Ref.’s mm for 3D TKIs, which did not 
consider the role of RKKY interactions. 


B. Methods 

We would like the study our fully interacting model at 
the mean field level. This means we should decouple our 
Kondo and RKKY interaction terms into the appropriate 
channels. This amounts to an assumption, which must be 
checked self-consistently, about the type of order in the 
interacting groundstate. To this end, we assume that the 
Kondo interaction, 

Si * Si — / ) (Ja.p * ^ 7 (5f-'m,a: ^Vn ,P d, fn / ~ ; d rt ,'.fi (T) 

771,771' 

induces a nonzero expectation value for the opera¬ 
tor {Cj di tPt(r ), and the RKKY interaction causes a 
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nonzero expectation value for the magnetic order param¬ 
eters (Si). 

We can now perform the standard mean-field analysis 
by decoupling the interactions into the channels 

jcUJpCp + 

j m (s x s x + s y s y ) 2 j m (s x )s x - j m (s x ) 2 ( 8 ) 

Where a and fi denote the two types spin-orbit coupled 
single particle states. 

Notice that the interaction in Eq. (El) contains terms 
like C\_d-C+d_, but these terms vanish after the above 
substitution since the resulting c-d hopping process does 
not conserve angular momentum. Also, we assumed that 
the spin-spin interaction favors magnetic order in the XY 
plane. We will justify this assumption in the next section. 

We are therefore left with a non-interacting Hamilto¬ 
nian for which the parameters (f >+, (f )_ and ( S x ) must be 
determined self-consistently. We use the parameters /i 
and A in Eq. © to enforce the conditions that there are 
2 electrons per d-site and one electron per graphene site, 
ensuring that the system is a band insulator when there 
is a full gap at the Fermi energy. 

Performing these substitutions, the MF Hamiltonian 
becomes 

H = CfcC^C/c + A {dlpijdipa — 2) + ^^^X C ia CilT ~ 

fc,t 7 i,P>< 7 i,cr 

"bA y^(d_|_ Q <r ctpd+p d_ a <j a pd^p) 

i 

-J 5^(0+ + 0-)(C ipadipcr + h.C.) 

i p,(j 

-2 J m Y,(S X )(dl t d +i + dl f d_ 4 . + h.c.) 

i 

+ J(4>+ + 4>-) 2 + Jm(S X ) 2 ( 9 ) 

We then diagonalize the mean-field Hamiltonian and 
write the bare electron operators in terms of the free 
excitations of the system 

A = Va (a<r) a t 

“fc.acr ^ u k.n U k,n 
n 

4, a cr = YjPk 0 !n a l,n’ ( 10 ) 

n 

which are linear combinations of c and d electrons. Since 
H is diagonal in the a) n operators, knowledge of the co¬ 
efficients a n and /3 n allow us to numerically determine 
the values of (C±d±), (5), (d\di) and (c|cj). By averag¬ 
ing over all occupied states of the mean-field model we 
can then find the parameters of H where the mean field 
constraints are satisfied. 

In general, for each order parameter, (X) there are two 
self-consistent solutions. These are (X) = 0 and (X) = 
i / 0. The solution with a nonzero order parameter 


breaks a symmetry of the interacting Hamiltonian and 
is therefore a distinct phase from the case where (X) = 
0. Furthermore, notice that the mean field Hamiltonian 
H = H°(J X x ) is only a function of the product J x and 
x. Therefore, for any value of x there will always be a 
value of J x such that ( X) = x. At this J x there are two 
solutions to the mean-field conditions, x = (X) ^ 0 and 
x = 0. 

The mean field solution is equivalent to the lowest 
energy noninteracting variational state |x). E V ar = 
(x\H\x). E var is a local minimum for the eigenstate \x) 
of H aif when the self consistency equation x = ( X ) is 
satisfied. In this case (x\H\x) = (x\H MF \x). This argu¬ 
ment allows us to compare the energies of the different 
self consistent solutions. Since these energy of solutions 
equals the variational energy, the solution with the low¬ 
est energy must be the best mean field approximation to 
the true groundstate. 

If the order-disorder transition is continuous, then ev¬ 
ery solution with nonzero x must be of lower energy than 
the disordered solution where x = 0. However, near a 
first order transition, there will exist solutions with small 
(. X ) that are of higher energy than the disordered state 
( X) = 0. This gives us an easy way to distinguish be¬ 
tween the two types of phase transitions in our calcula¬ 
tion. 

Note that model © does not conserve particle hole 
symmetry, whereas the fully interacting model ([6]) does, 
so that in order to ensure there are the same number of 
conduction electrons we need to adjust /x so that each 
site on the graphene lattice is filled to n c = 2. 


C. Phases 

There are three distinct phases in model (pi), which 
occur when a) (C^d) = 0, b) (C' f d) ^ 0 and (S) = 0 or 
c) (Ctd) ^ 0 and (S) ± 0. 

The first case occurs when (C\_d + ) = (Clc?_) = 0 and 
there is no Hondo screening of the d-electrons. This is the 
“Fully Polarized” or Magnetic phase. This phase may be 
considered uninteresting as there is no interplay between 
the c and d electrons, which know nothing about each 
other at the mean field level. The conduction electrons 
are thus completely noninteracting and form a semimetal 
exactly like graphene with a pair of gapless Dirac cones in 
the band structure. The d-electrons still contain the spin- 
orbit term and interact through the spin-spin interaction 
J m . In principle the spins could order in any number of 
phases depending on the exact form of the Heisenberg 
interaction, however the most likely result is that they 
order magnetically. 

Now, if the spins ordered magnetically in the z- 
direction, the S.O. term favors the state |+ J,, — f), while 
the J m term favors the state |+ f, — t)- These states are 
completely orthogonal, therefore the spins will order in 
one of these two states depending on which interaction 
is stronger. If A > J m , the first state will be chosen and 
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the energy per site is E\ = —A. On the other hand if 
J m > A, the spins will align in the S z = +1 state and 
the energy per spin will be Ez = —J m . On the other 
hand, if the spin order in the XY plane, the spins order 
depending on the Hamiltonian 


H d = v* 


A J m S x 0 0 

J m s x -A 0 0 

0 0 -A J m S x 

0 0 J m S x A 


where v = ( d+^d+^d-^d-^ ). 


( 11 ) 


The energy of the groundstate with n d = 2 is Exy = 
— h 2 + ( J m S x ) , where the self-consistent value of 
S x = (S x ) depends on the ratio J m /A. Clearly, by the 
variational principle, Exy < Ez and Exy < E\ for all 
A and J m . So ordering in the XY plane allows the spins 
to partially satisfy both the S.O. and RKKY terms in 
the Hamiltonian, so that we can safely assume that the 
spins order this way. Therefore, in the fully polarized 
phase the c and d electrons behave independently, with 
the d electrons ordering magnetically according to the 
Hamiltonian in 

The second phase we consider occurs when (C±d±) ^ 0 
and ( S) = 0. We will call this phase the topological 
Kondo insulator (TKI), or simply the Hondo phase. The 
nonzero value of <fi causes the c and d electron states to 
hybridize. Since ( S) = 0, the RKKY interaction has 
no effect on the mean-field groundstate. This regime 
can be thought of as the many-body analog of the sin¬ 
gle impurity Kondo problem. In that problem when a 
single magnetic impurity is immersed in a metal, at low 
enough temperatures the magnetic moment is screened 
via the formation of a singlet pair with the nearby con¬ 
duction electrons. In our model, there is a dense lattice 
of magnetic moments, and every moment is completely 
screened by the nearby graphene conduction electrons. 
In ref. [T9] , it was shown that such a phase is the sta¬ 
ble solution to the Kondo lattice problem in the large N 
limit, and that in this limit the phase is indeed equivalent 
to a conduction sea screening a dilute set of magnetic im¬ 
purities. The result of the screening at each lattice site 
is that the conduction electrons and local moments hy¬ 
bridize to form a single composite object. Therefore the 
Kondo phase is a free electron system where the resulting 
free electrons are linear combinations of the bare c and d 
electrons and are the eigenstates of |9| with ( S x ) = 0. 

The hybridization between the c and d electrons opens 
a gap at the K and K' points in the graphene band 
structure (that is at the gapless nodes in the Brillouin 
zone). However, the form of the coefficient, V±(fc), en¬ 
sures that there is no hybridization at k = 0. Therefore, 
when A = 0, the d-electron like band in the band struc¬ 
ture remains gapless at k = 0. However, a nonzero A 
opens a gap at the B.Z. center and creates a full band 
gap. The presence of the spin-orbit coupling also gives 
the band structure a nontrivial topology. This can be 
seen by calculating the parity eigenvalues at the special 


time-reversal invariant momenta in the B.Z.. According 
to ref. [20], a topological invariant v can be defined by 

(-ir = n^) (i2) 

3 =1 

whereby v = 0 corresponds to the topologically triv¬ 
ial phase and v = 1 the nontrivial phase. The parity 
eigenvalues can be calculated directly by diagonalizing 
the mean-field Hamiltonian. This was done in Ref. [8] 
and we have repeated this calculation, verifying that the 
TKI phase is indeed topologically nontrivial. We have 
also solved the model self-consistently on a finite width 
strip with armchair edges, and have shown that in the 
Kondo phase when A ^ 0, at every energy within the 
band gap there exists a single Kramers pair of gapless 
edge states. In section III, we will study the effect of the 
RKKY interaction on these edge states and comment on 
the possibility there exists a phase where time-reversal 
symmetry is broken on the edge while preserved in the 
bulk. 

The final, distinct, phase that can occur is one where 
there is incomplete Kondo screening of the d moments, 
so that there sis a mixture of magnetic order and Kondo 
order. It is important to note the previous Kondo phase 
is only a mean-field state and that the fully interacting 
many-body phase contain additional correlations. In our 
model we deal with these correlation by including the 
RKKY term in our Hamiltonian. That is, both (j> = 
(CU) and ( S) are nonzero in this phase. In this case, 
time-reversal symmetry is broken everywhere in the bulk. 
In the presence of such spontaneous symmetry breaking 
there is no distinction between the trivial band insulator 
and the topological insulator phases. 


D. Phase Diagram and Transitions 

In this section we give the results of our mean-field 
calculation for the bulk system. 

Kondo Order- We start with the simplest case, where we 
set J m = 0 so that spin-spin interactions do not compete 
with the tendency for Kondo order. However, even this 
simplest case shows interesting physics. First studied by 
Withoff and Fradkin in Ref. EH, who showed that the ef¬ 
fect of a vanishing density of states in the band structure 
creates a critical point J c below which the Kondo effect 
does not take place. Careful renormalization group cal¬ 
culations on the spin-f pseudogap Kondo model 22 ! have 
since shown that, in the case of graphene, there are signif¬ 
icant corrections to the large-N result near such a critical 
point and that these results depend on the presence or 
absence of particle-hole symmetry in the modeP^D. 

Our mean field calculation, however, is similar to other 
large-N studies of the pseudogap Kondo problenPH. Here 
the large-N mean field approach is used in order to ac¬ 
curately describe the physics within the Kondo phase, 
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FIG. 1: a) The Kondo order parameter for three values of A, ranging from A = 0 to A = 0.5 (left) and the corresponding 
energy gain compared to the 0 = 0 solution (right) . When A = 0 the transition is continuous but becomes first order when 
A 0 0. b) The magnetic (left) and Kondo (middle) order parameters for fixed J = 0.3, A = 0.5. The onset of magnetic order 
with increasing J m shows the transition from the Kondo phase into the mixed phase. Further increasing J m to the point where 
the Kondo order parameter drops to zero this signals the transition into the fully polarized phase. The energy crossover of this 
mixed phase with the fully magnetic (top-right) and TKI (bottom-right) phases show that both these transitions are first order. 


at the cost of not being able to accurately describe the 
critical point of this phase transition. In the following 
we will describe the complete solution of the mean-field 
calculation, including all phase transitions. We should 
keep in mind, however, that the quantitative details of 
any continuous transition are valid only as N —► oo and 
are not expected to hold beyond the mean field level. 

When A = 0, all d-electron sites are equivalent 
and there is only a single order parameter (C\d + ) = 
(Cld_) = <p. In Ref. [2T] it was shown that the ef¬ 
fect of a non-constant density of states leads to a critical 
J c = , which signals the onset of Kondo order in this 

model. 

We will see how this works in the mean field calculation 
of our graphene model. Consider the Lagrangian form of 
Eq. (J9), 


Integrating out the fermions, v = (cA,CB,d+,d-), pro¬ 
duces the generating function, 


Z = exp 


dui 


—dk I a ( Hap (k) - iui6 a p ) Ip 


■ (14) 


The desired correlation functions can then be derived 
from the generating function using the expression 



dkdui (G a p(k,iu])) 


S 2 Z 


Sljlg 


1=0 


(15) 


£ = / duidk [t)l (' H™p(k) - iui5 a p)vp + I a v a + v^ a I a 


This is analytically tractable for small values of (f>±, 
where we can easily take the inverse in Eq. (14) and throw 
(13) out all terms of 0(</> 3 ) or greater. This gives 


(cU+) 

(cU-) 

(dU±) 


dkdui 

dkdui 


J(0 + + 0_)(K + C e -K!) 
(A + A - iui)(-\f(k)\ - iui) 
J(0 + + 0_)(K_e» e ~^) 
(A - A - iui)(-\f(k)\ - iui) 


dk / dui 


(I/I -*w)(-|/| -*<*>) 


(A ± A - iw)(|/| - iui)(-\f\ - iui ) + 6i</> 2 (-|/| - iui) + fo 2 ^ 2 (|/| - iui) 
I 


where we have written the graphene dispersion as f(k) = energy. 
\f{k)\e ie ( k ' > . We perform the ui integral by contour in¬ 
tegration and exchange the momentum integral for an 
integral over energy, f kdk = p^ede, where D is the 
bandwidth and po is the density of states near the Fermi 


(16) 


The conditions (C]_d + ) = 0 + , (Cld_) = </>_ and 
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(cftd) = 2 lead to the three equations 


E. Magnetic Order 


(A - A) log 
(A + A) log 


A-A 
D + A-A 
A + A 
D + X + A 


~ A - A 


D+ PoJ( l+ <t>+/<!>-) 

D+ p 0 J {! + <!>-/<!>+) 
(15) 


where we ignored the constant factor from the k depen¬ 
dence of V± and 9, and the third relation comes from 
looking at the shift in the residue iuj due to small but 
nonzero values of A and qt>. 

When A = 0, then <^ + /^_ = 1. In this case there only 
exists a solution for A when J > -A=- Near J c = -A=- 
Eq’s ( |15| ) give the solution A = J — J c . Therefore, <j> = 
(C^d) = \JJ — J c , and there is a second order transition 
at J — J g . 


When there is a nonzero spin-orbit coupling A / 0, 
the ratio 4 > +/4 > - ^ 1) changing the solution to the first 
two equations in (15). Therefore, the value of the critical 
J for one of these equations is increased while the other 
decreases. Since both equations must be satisfied the 
overall effect of A is to push the transition back to larger 
values of J C (A) > J c (0). 


We verify these results is Fig. [T| through the method 
detailed earlier. We see that our results agree with the 
general argument just presented. In particular, we see 
that when A = 0, 0 decreases continuously to zero im¬ 
plying that there is a second order transition at a finite J c 
between the phase with no Kondo order and the Kondo 
phase. 


We also consider case where A / 0, where we show 
the numerical data for A/t = 0.1 and 0.5. As expected, 
we now see a splitting between the <j>+ and <j>- order pa¬ 
rameters. We also see that the phase transition becomes 
first order. In fact we can always find a self-consistent 
solution for any value of the order parameters down to 
<j>± = 0, but as shown in Fig. [T] these solutions are of 
higher energy than the disordered phase. This is easy to 
understand, as a finite A lowers the variational energy 
of the fully polarized phase more than it lowers the en¬ 
ergy of the Kondo phase. This is because d-electron-like 
bands in the Kondo phase are necessarily linear combi¬ 
nations of both d+ and d_ electrons, while in the fully 
polarized phase every d-site is filled with c?_ electrons. 
This causes an energy crossing between the two possible 
phases, which occurs away from the critical point. This 
pushes the fully polarized phase into Kondo regime as 
you increase A, and causes a first order transition. Fig. [2] 
a) shows the phase diagram in the J—A plane for J m = 0. 
The transition at A = 0 is continuous with the critical 
point described above, while for A / 0, the transition is 
driven first order. 


Next, we ask what happens when we include the spin- 
spin interactions between the d-electrons. In this case, 
there is an interplay between three competing forces, the 
spin-orbit coupling, the Kondo interaction and the mag¬ 
netic interaction. Despite this competition, there exists a 
phase in which both Kondo and magnetic orders coexist. 
This is characterized by a nonzero value of both (C^d) 
and (S). Here we will look at the stability of the Kondo 
phase to both the fully polarized phase and this mixed 
order phase. 

First consider the fully interacting Hamiltonian. Deep 
within the Kondo phase, the mean field solution to 
H mf ((Ct d) 0) is a saddle point of the action. When 
A / 0, there is a band gap separating the highest oc¬ 
cupied band from the lowest unoccupied band so that 
there are no gapless excitations. We can then treat the 
spin-spin interaction as a perturbation around this solu¬ 
tion, but it will obviously have no effect if the interaction 
strength is smaller than the gap. At the mean held level, 
the energies of some occupied states will shift down while 
an equal number of states will have their energy shifted 
up. Since all states are a finite energy below the Fermi 
level due to the band gap, all these states will remain oc¬ 
cupied and to lowest order in perturbation theory there 
will be no effect. 

The situation is slightly less obvious when A = 0 and 
the band structure is gapless. In the Kondo phase the 
bands touch at the Fermi level only at the F point in the 
Brillouin zone. This gapless band touching is guaranteed 
by the fact that V±(k = 0) = 0. In this case, a perturba¬ 
tion on the J m = 0 MF solution will raise the energy of 
some states near k = 0 above e F - The change in energy 
due to this shift is given by ~ J(«S), while the number 
of states is limited to ~ J(S)N(eF )• Meanwhile, from 
H mf , such a perturbation has a constant energy cost of 
Jm(S) 2 - To first order, the total change in energy would 
be 


A E ~ - Cl (J m (S)) 2 N(e F ) + J m ((S)) 2 . (15) 

Since N(€f) —> 0 , it seems likely that even in the pres¬ 
ence of gapless states near the T point, a perturbation in 
J m will have a very small effect on the Kondo phase. We 
verify both these claims by explicit calculation. 

Fig. 0 b) shows this behavior for a cut in the phase 
diagram along J = const and A = const. For J m small, 
there is no magnetic order and the Kondo phase is sta¬ 
ble. If the gap to single spin excitations is smaller than 
the energy cost of destroying the Kondo phase, then the 
mixed phase is stable for some regime of J m . The nu¬ 
merical calculation shows the spin order parameter (S) 
turning on for some finite J m and coexisting with the 
Kondo order parameter <j>. As the spin-order increases 
there is a corresponding drop in the Kondo order param¬ 
eter. At some points in phase space, for small J, the 
energy of the fully polarized phase is always lower than 
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FIG. 2: Cuts showing the 3D phase diagram, a) The J — A plane when J m = 0. b) The J — J m plane when A = 0. c) The 
J — Jm plane when A = 0.5. See text for the description of the three phases. 


the energy of the mixed phase. In this case, there is a 
direct transition from the Kondo phase to the magnetic 
phase. The energy crossings in Fig. [I] b) show that the 
transitions both into the mixed phase and into the fully 
polarized phase are first order. 


F. 3D Phase Diagram 

Fig- 0 shows 2D cuts of the phase diagram at J m = 0 
and at two different values of A. Taken together these 
gives us the full 3D mean field phase diagram of our in¬ 
teracting model. 

When J m = 0, there is no competing magnetic inter¬ 
action. At A = 0 there is a second order transition into 
the Kondo phase at J = J c . As discussed, this critical 
point is a result of the vanishing density of states near 
ep- In the non-Kondo phase the d-electrons have no or¬ 
der but are unstable to any infinitesimal interaction. For 
A / 0, the d-levels are split and a full gap opens in the 
Kondo band structure which gives the phase a nontriv¬ 
ial topology. Here, the transition between the polarized 
phase and the Kondo phase is driven first order. 

We also show cuts in the J — J m plane along A = 0 
and A = 0.5. The main difference between these is that 
when A/0, Kondo phase is destroyed suddenly at small 
J. Further, when A ^ 0 the Kondo phase has a full band 
gap with a nontrivial topology, while the Kondo phase 
for A = 0 is a semimetal. The similarity of the two cuts 
is due to the stability of the Kondo phase against J m in 
both cases. At the points in phase space where there is 
nonzero magnetization, J m is generally large enough that 
the effect of A only slightly moves the boundaries. 


III. EDGE STATES OF THE TKI PHASE 

Perhaps the most dramatic consequence of symmetry 
protected topological states is the necessary existence of 
nontrivial edge states in systems with a boundary. In two 
dimensions the allowed edge states are either A) gapless 
or B) spontaneously break the symmetry, while the sys¬ 
tem remains gapped with unbroken symmetry within the 
bullP^. In three dimensions a third allowed possibility is 
a surface with topological order 2 -It would be very 


interesting if such a surface state could be achieved with 
a topological Kondo insulator, however in our 2D system 
we must restrict ourselves to the first two possibilities. It 
has been shown that on the surface of 3D TKIs, fluctua¬ 
tions around the mean-field state can lead to strongly in¬ 
teracting surface theorie^. We take a similar approach 
below for 2D TKIs where we first discuss the mean-field 
solution and then go beyond MFT to look at the true low 
energy theory of our edge states. 

A. Mean Field Analysis 

We now show that at the mean field level, when the 
system is studied on a finite strip, so that the noninteract¬ 
ing TKI phase contains gapless edge states, the edges are 
always unstable to magnetic perturbations at the Fermi 
level. In this way, the TKI is a simple way to realize a 
time-reversal invariant insulator with spontaneous break¬ 
ing of the TR symmetry on the edges. 

In the bulk system, deep within the Kondo regime, the 
effect of an infinitesimal RKKY interaction is negligible 
due to the presence of a gap to any spin-1 excitations. 

Our argument for the MF edge theory is similar to 
the Peierls argument 31 33 whereby in one dimension, if 
we ignore the dynamics of the phonons, logarithmically 
more energy can always be gained by the electrons or- 
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FIG. 3: The energy gained by opening a gap at the Fermi level 
near k = n is logarithmic in JS in the mean-field calculation. 
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FIG. 4: (top) The single-particle band structure on a finite 
strip shows in gap edge states and a gap opening in the edge 
spectrum for small J m . (bottom) Measuring the symmetry 
breaking shows that the magnetic order is localized near k x = 
7r (left) , and at the edges of the strip y = 0 ,L V (right), 
(y = 0 and y = L y /2 gives the largest and smallest peaks 
respectively.) 


dering. The mean field (S) plays the role of the static 
phonon fields, and the periodic lattice distortion is in¬ 
stead replaced by a coupling between right and left mov¬ 
ing electron fields. The effect of the magnetic interac¬ 
tion is to open a gap at the Fermi level, replaci ng the 
edge spectrum e k ~ v k k with e k ~ ( Vkk ) 2 + A 2 where 
A = J m (S). Following the standard argument, one di¬ 
mensional gapped systems contain a singularity in the 
density of states, and a proper estimate for the change in 
energy due to the magnetic ordering is 


SE ~ f dk(^/(v k k) 2 + A 2 — Vkk) = ^-A 2 log[4eA 2 /A 2 ]. 

Jo 4 

(15) 

We verify this argument with an explicit mean-field 
calculation in the same vein as in section II for the bulk 
phase diagram. We perform the mean-field calculation in 
the same way as for the bulk system. The Hamiltonian 
now has an 8 N y site basis, where N y is the number of 
unit cells of our finite strip in the y direction. We use 
different order parameters for sites on the edge and in 
the bulk and find the self-consistent values of these order 
parameters. For the bulk order parameter we average the 
order over all sites that are not on the edge. In Fig|4j we 
have tuned J and A to a point deep within the Kondo 
phase. We saw in the previous section that this phase 
is extremely robust against the formation of magnetic 
order. An important point is that in order to satisfy the 
conditions (d)d) = (c^c) = 2 on every site, the edge states 
need to intersect the Fermi level, cf, at exactly k = n. 
This is the time-reversal invariant momentum at which, 
due to Kramer’s theorem, the two edge states intersect. 
In the mean field, the spin-spin interaction term couples 


fermions of opposite spin at the same momentum. At k = 
7r, these two fermions modes are degenerate with energy 
e = ep. The spin interaction breaks this degeneracy, 
sending one state above cf and one below ep- Consider 
the Hamiltonian describing of the edge states near Uf, 


Hedge — I 

— A<k— 7r<A 


J J 


€-k 'Jm 
Jm e k 



(15) 

The eigenvalues of H are e± = \]e'j, + J 2 ,. When 
£fc "C Jm, the corresponding eigenstates are given by 
v± = Cfcf ± Cki + O(j). Meanwhile, for J -C e k , to 
first order the eigenstates are just the original states 
V+ = Cfc f + ^Cki and v _ = c*,; - In the first 

case, the conduction electrons are almost completely po¬ 
larized in the XY plane, while in the second case the 
eigenstates have very little magnetic order. Therefore, it 
is only in the regime where e k <C J m , that a small J m 
creates a significant magnetization. But since the edge 
states are gapless there will always be some finite region 
where this condition is true, and it is these small number 
of states which contribute to the spontaneous breaking 
of TR symmetry on the edge. 

In Fig. [3j we verify that the energy of a self-consistent 
solution of the MF Hamiltonian as a function of the input 
parameter JS is 


A E ~ (J m S) 2 \ log(J m S/A)| - J m S 2 (15) 


where A is an arbitrary cutoff. Therefore, for any value 
of J m , there is an S for which the energy gain is positive, 
making this TR broken state favorable. Since the self- 
consistent point S = ( S ) is the variational minimum, it 
must therefore also have a positive gain in energy. Fig. [4] 
c) and d) show that the symmetry breaking is localized to 
momenta near the TRIM k = it and is localized in space 
to the edge of the strip. Meanwhile, Fig.’s [4] a) and b) 
show the band structure of the finite strip. In particular, 
they show the opening of a gap at the Fermi level due to 
a small nonzero J m . 


B. Luttinger Liquid Physics 

The Peierls argument and mean-field calculations in 
the previous section are only valid in the limit of a static 
spin field (similar to how the lattice Peierls argument 
is only valid for static phonons). For a one-dimensional 
system, however, we can solve the low-energy theory ex¬ 
actly using bosonization techniques. In gapless ID sys¬ 
tem, the low-energy excitations are bosonic degrees of 
freedom which are localized near the Fermi points. For a 
non-interacting topological insulator it is well known that 
each edge contains a time-reversed pair of chiral fermion 
inodes 4 ^, for example a right-moving spin up mode and a 
left-moving spin-down mode. Taken together these com¬ 
prise a single bosonic degree of freedom, so that the prob¬ 
lem maps onto a spinless fermion problem. In the absence 











9 


of interactions, which may couple the right and left mov¬ 
ing modes, the bosonic action for the edge of our TKI is 
just the action of a free bosonic particle 


S = Vk dxdr 


K(d x <f>) 2 + -(dr *) 2 


(16) 


In our case, the neutrality condition 

(17) 

k k 


ensures that the edge states cross the Fermi level at ex¬ 
actly k = 7r. Near this Fermi level, the right and left mov¬ 
ing modes can be bosonized to give a single free bosonic 
degree of freedom which describes the low-energy dynam¬ 
ics of the system. This allows us to give a rather simple 
interpretation of the mean-field result in terms of the al¬ 
lowed interaction between the low-energy modes of our 
theory. We will further see how the mean field result 
fails to properly capture the fluctuations of the ID edge 
modes. 

At the Fermi point we have a single right moving spin 
up mode and a left moving spin down mode. The spin- 
spin interaction, J m , couples these modes together. We 
can expand the fermion creation and annihilation oper¬ 
ators in terms of bosonic operators. The most relevant 
terms in this expansion are 

VV-eW). (18) 


At the mean field level, the spin-spin interaction is 
J m (S)S x , where S x = + cft_^d-± + h.c.). r tp R 

is a linear combination of all spin up c and d operators, 
and likewise for ipL and spin down operators. There¬ 
fore, writing this interaction in terms of ipR/L produces 
a number of terms. The most relevant allowed terms is, 


3mf (V'flV’L + V’lV’i?) ~ cos(23), (19) 


where g MF ~ J m (S). The scaling dimension of such an 
operator is well known from the form of the free bosonic 
correlation function to be 

dim [cos (p3)] = ^ , dim [cos(p^)] = ^-. (20) 

Therefore the operator cos (26) is a highly relevant pertur¬ 
bation at the noninteracting point K = 1. This implies 
that for the mean-field model, an infinitesimal pertur¬ 
bation J m will flow under RG to strong coupling. The 
resulting model is a sine-Gordon model, in which the 6 
field is pinned at strong coupling, breaking TR symme¬ 
try. 

However, this type of naive analysis ignores the fluctu¬ 
ations of the spin order parameter (S), which can drasti¬ 
cally affect the physics. In particular, we can write down 
all allowed interactions involving a single right and a sin¬ 
gle left moving mode. These interactions are 

H int = gi^R^R^L^L (21) 

H int = gz^R^R'I’Lll’L (22) 


where the umklapp operator g 2 , can only exist in systems 
with a single fermion species if there is point splitting 
g 2 ~ tl}' R (x)i/)' R (x + a)ip' L (x)ipL{x + a )■ This introduces a 
derivative upon taking the continuum limit, making this 
operator less relevant. 

The most relevant bosonized expressions contained in 
Eq.’s (21) and (22) are 


~ 3i [( d - ( d x 6 ) 2 ] (23) 

H?'l ~ 32 cos(43). (24) 


The gi term can be absorbed into the action, Eq. ( |T6| , 
by renormalizing the value of the Luttinger parameter 
K. The g 2 term, however, will attempt to pin the field 3, 
thus opening a gap in the energy spectrum at the Fermi 
level. This process is relevant if the scaling dimension for 
the g- 2 operator is less than 2. In that case, the cosine 
operator will flow to strong coupling and pin the 9 field. 
By Eq. ( [20| , the scaling dimension of the g 2 operator is 
dim [g 2 ] = 4/AT. This implies that at the noninteracting 
point K = 1, the cosine term is irrelevant and the action 
of the edge states remains gapless. It is only when the 
gi term pushes the value of K past K = 2 that the 
cosine operator becomes relevant. At this point there is 
a Kosterlitz-Thouless transition into a phase where the 
9 field is pinned. The 9 field in this case acts like the S x 
spin operator since, by the bosonization rules above 


S x ~ = ^r^l + ^ l ^r = cos(2 9). (25) 


Pinning 9 at 9 = 0, implies that ( S x ) ^ 0, and so time 
reversal symmetry is broken on the edge. 

The stability of edge states in the spin-quantum Hall 
effect has been studied previously, focus ing on the effect 
of a screened Coloumb interactior j 34 * 35 ) The results in 
those works similarly find that a single Kramer’s pair 
of edge modes are stable at weak coupling, but may be 
driven into a gapped phase by sufficiently strong inter¬ 
actions. In our problem, the analogous interactions are 
included naturally in the form of the RKKY interaction 
term. 

This result leaves two possibilities for the full phase 
diagram in the presence of edge states. The first case is 
that the strength of RKKY interactions, J m , required to 
gap the edge modes for a given value of J is less than 
that required to drive the edge into a magnetic phase. In 
this case, time-reversal symmetry will be broken sponta¬ 
neously on the edge of the system while being preserved 
within the bulk, and the edge properties of the TKI in 
this phase will differ dramatically from that of the un¬ 
correlated 2D topological insulator of Ref’s |9I12| . 

The second possibility is that for all J, the required 
J m to drive the edge to a gapped state is greater or equal 
to that required to drive the bulk into the magnetic or 
mixed phases. In this case, the low energy theory of 
the TKI is qualitatively similar at weak coupling to the 
noninteracting TKI phase of Ref. [S]. However, even in 
this case, the low-energy edge theory, while gappless, is 
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still governed by the action in Eq. (16) and controlled by 
the Luttinger parameters vf and K and thus will show 
quantitative differences from the noninteracting theory. 

We may also ask, in what case is the mean-field anal¬ 
ysis of the previous section valid. If we generalize our 
model to one with Nf flavors of fermions on each lattice 
site we enter a regime where the larg e-Nf and slave- boson 
approaches to the Kondo problem become justified- 17 * 19 ! 
In the limit Nf —> oo, the mean-field result becomes ex¬ 
act as fluctuations, even on the edge, are strongly sup¬ 
pressed. Within the Luttinger liquid framework, we now 
need to study a system with Nf time-reversed pairs of 
gappless modes on each edge. In the topologically non¬ 
trivial phase, Nf must be odd. It is mentioned in Ref. l84l 
that the Nf = 3 case is less stable to interactions than 
the Nf = 1 case. This is due to the enlarged number 
of allowed interactions which appear when we include 
terms which couple different modes together. We there¬ 
fore spectulate that as Nf —>■ oo, the huge number of al¬ 
lowed interactions in the low-energy picture will always 
gap out all edge modes in order to agree with the high- 
energy mean field picture. The large- Nf theory should 
always fall under the first case discussed above where TR 
symmetry is broken on the edge but preserved within the 
bulk. 


IV. CONCLUSIONS 

In this paper, we have discussed the role of interac¬ 
tions in the physics of 2D topological Kondo insulators. 
To this end, we wrote down a realistic theory for a sys¬ 
tem of graphene doped with 5d adatoms where there are 
two localized d-electrons on each adatom which form a 
composite spin-1 magnetic moment and interact with the 
nearby conduction electrons on the graphene lattice. We 
performed a mean-field calculation on this model where 
we were careful to note that included within the Kondo 
interaction is an RKKY type spin-spin term between the 
localized d-electron states, which appears at second order 
in perturbation theory in this interaction. Including both 
the Kondo and R.RKY couplings as separate parameters 
allows us to decouple our theory into both the Kondo 
and magnetic ordering channels. 

First, we found that the bulk phase diagram is sep¬ 
arated into three distinct regions. For sufficiently large 
values of the Kondo coupling and weak RKKY interac¬ 
tions the Kondo insulator phase is stabilized and posseses 


the same low-energy theory as a noninteracting topologi¬ 
cal insulator. As the relative strength of the RKKY term 
is increased the localized moments break time-reversal 
symmetry by either becoming fully polarized or enter¬ 
ing a mixed phase with both magnetic and Kondo order. 
The nontrivial topological distinction of the TKI phase 
is completely lost in both of these magnetic phases. 

The TKI phase is fully gapped and so we expect that 
any fluctuations around the mean-field result can be ne¬ 
glected. Therefore, to look for nontrivial correlation phe¬ 
nomena we further studied the effects of the Kondo in¬ 
teraction on the gappless edge states of this topological 
phase. 

We find that in the large Nf limit, where the mean 
field treatment becomes exact, a gap is always opened in 
the edge spectrum and TR symmetry is spontaneously 
broken locally at the edges. At weak coupling, when the 
RKKY parameter J m is small, there exists a critical l/Nf 
which marks a qualitative change in behaviour whereby 
strong fluctuations at the edge destroy the magnetic or¬ 
der of the MF solution. For Nf = 1, as in our original mi¬ 
croscopic model, this leads to a phase with gappless edge 
states which are described by the Luttinger liquid Hamil¬ 
tonian with interaction dependant Luttinger parameter 
K. This 2D TKI therefore provides a natural system 
where interactions cannot be ignored if one is to properly 
describe the edge phenomena quantitatively. At interme¬ 
diate values of the coupling J m , an edge transition into 
the gapped phase occurs independantly of the magnetic 
transition in the bulk which may lead to a qualitative 
deviation from the noninteracting edge theory. Indeed, 
there two possibilities within the TKI phase at interme¬ 
diate coupling. The first is that the edge states remain 
gappless everywhere in this phase. The second is that 
there is a transition into a phase with magnetic order 
on the edge but not in the bulk. While we cannot rule 
out either of these two cases within our analysis, the sec¬ 
ond possibility represents a dramatic departure from the 
edge theory of the noninteracting topolgical insulators 
and thus is a possible avenue to search for correlation 
effects in topological phases. 
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